Papers with Extractive Question Answering tasks
Bootstrapping Small & High Performance Language Models with Unmasking-Removal Training Policy (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models (LMs) have shown promising ability on handling various downstream tasks including textual classification and question answering. |
| Approach: | They propose to use BabyBERTa to train child-directed speech without unmasking words while masking parameters to improve grammatical accuracy. |
| Outcome: | The proposed model achieves grammatical ability comparable to RoBERTa-base model, which is trained on 6,000 times more words and 15 times more parameters. |